Multi-operator image retargeting with automatic integration of direct and indirect seam carving

a r t i c l e i n f o Multi-operator image resizing can preserve important objects and structure in an image by combining multiple image resizing operators. However, traditional multi-operator methods do not take both horizontal and vertical content-aware resizing potential into consideration, which essentially leads to squeeze/stretch effect in the resultant images. In this paper, we propose a new multi-operator scheme that addresses aforemen-tioned issue by integrating direct and indirect seam carving. Compared with previous methods, the proposed scheme remarkably reduces the cost of deciding when to change operators, by employing a newly defined image artifact measure. Furthermore, we propose a novel seam carving enhancement, named ACcumulated Energy Seam Carving (ACESC), as a basic operator to improve global structure preservation. By combining horizontal and vertical seam carving, our scheme preserves the shapes of important objects well. We present typical results to demonstrate the effectiveness of our method. User study shows that our method has high user preference. In recent years, content-aware image resizing (a. k. a., retargeting) techniques, which can preserve visually important contents and maintain good perceptual invariance in an image when the size and aspect ratio are changed, have evoked a great deal of interests. Such techniques are especially meaningful for transforming images or video clips between multimedia devices with different resolutions. The techniques can be coarsely divided into five major categories [7]: cropping methods, warping methods, seam carving methods, patch based methods and multi-operator retargeting methods. In cropping methods [8–11], an optimal sub-window of the target size, which contains visually important regions, is searched from the input image. However, a disadvantage is that such methods may discard a large part of essential regions when the important objects are near the image periphery. Warping methods resize images non-homogeneously [3,12–16]. By fixing a mesh on the image, they warp the designed mesh non-uniformly to reach the desired size based on a global optimization function. Patch based methods [17] achieve resizing by optimizing a patch-based similarity measure between the input and target image. One limitation of this method is its high computational cost, as pointed out in [6]. Seam carving (SC) methods [4,18] perform image retargeting by iteratively adding or removing the most unimportant curves (i.e., seams) from the image. For images with large homogeneous region going across from its left to right (e.g., the sky region in Fig. 1 going across the image from left to …